H.264/AVC, one of standard specifications for image encoding schemes, can use different quantization steps for orthogonal transform coefficient components to quantize image data in a profile equal to High Profile or higher. A quantization step for each orthogonal transform coefficient component can be configured based on a quantization matrix (also referred to as a scaling list) and a reference step value. The quantization matrix is defined as a size substantially the same as an orthogonal transform unit.
FIG. 19 illustrates preset values (default values) for four types of quantization matrices predefined in H.264/AVC. For example, matrix SL01 is a default for the quantization matrix if the transform unit size is 4×4 in intra prediction mode. Matrix SL02 is a default for the quantization matrix if the transform unit size is 4×4 in inter prediction mode. Matrix SL03 is a default for the quantization matrix if the transform unit size is 8×8 in intra prediction mode. Matrix SL04 is a default for the quantization matrix if the transform unit size is 8×8 in inter prediction mode. A user can use a sequence parameter set or a picture parameter set to specify a specific quantization matrix different from the default values shown in FIG. 19. If the quantization matrix is not used, an equal value is used for all components of the quantization step used for the quantization.
High Efficiency Video Coding (HEVC) is a next-generation image encoding scheme as a successor to H.264/AVC and its standardization is promoted. HEVC incorporates the concept of coding unit (CU) which corresponds to a conventional macro block (see Non-Patent Literature 1 below). The sequence parameter set specifies a range of coding unit sizes using a set of power-of-two values which are a largest coding unit (LCU) and a smallest coding unit (SCU). The use of split flag specifies a specific coding unit size within the range specified by LCU and SCU.
According to HEVC, one coding unit can be divided into one or more orthogonal transformation units, namely one or more transform units (TUs). The transform unit size can be set to any of 4×4, 8×8, 16×16, and 32×32. Accordingly, a quantization matrix can be specified according to each of these transform unit size candidates.
H.264/AVC allows for designating only one quantization matrix for one transform unit size within one picture as specified in the released reference software (http://iphome.hhi.de/suchring/tml/index.htm) referred to as a joint model (JM). By contrast, Non-Patent Literature 2 shown below proposes to designate multiple quantization matrix candidates for one transform unit size within one picture and adaptively select a quantization matrix for each block from the viewpoint of rate-distortion (RD) optimization.